DOCUMENT RESUME 



ED 466 691 



TM 034 270 



AUTHOR 

TITLE 

PUB DATE 
NOTE 



PUB TYPE 
EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



Lubliner, Shira 

The Power of Clarifying: A Comparative Analysis of Strategies 
-That Strengthen Comprehension. 

2002-04-00 

26p. ; Paper presented at the Annual Meeting of the American 
Educational Research Association {New Orleans, LA, April 1-5, 
2002 ). 

Reports - Research (143) — Speeches/Meeting Papers (150) 

EDRS Price MF01/PC02 Plus Postage. 

^Cognitive Processes; ^Comprehension; ^Elementary School 
Students; Intermediate Grades; ^Learning Strategies; Skill 
Development; Social Studies; ^Teaching Methods 
Requests for Clarification (Responses) 



ABSTRACT 

This quasi-experimental study was designed to investigate the 
effects of cognitive strategy instruction on fifth-grade students' reading 
comprehension and comprehension of words encountered during reading. Eight 
intact fifth-grade classes of approximately 34 students each were randomly 
assigned to treatment groups (clarifying, questioning, combined questioning 
and clarifying, and control) for a 5-week study. Strategy instruction took 
place during social studies based on the belief that content area textbooks 
are the most difficult for children to read and comprehend (J. Chall, V. 
Jacobs, and L. Baldwin, 1990). Results indicate that the clarifying group's 
scores in reading comprehension and word comprehension were significantly 
higher than those in the control group, suggesting that clarifying strategy 
instruction positively affected fifth grade students' comprehension of texts 
and vocabulary encountered during reading. (Contains 5 figures, 4 tables, and 
54 references.) (Author/SLD) 



Reproductions supplied by EDRS are the best that can be made 
from the original document. 



TM034270 ed 466 



'O 



PERMISSION TO REPRODUCE AND 
DISSEMINATE THIS MATERIAL HAS 
BEEN GRANTED BY 



U.S. DEPARTMENT OF EDUCATION 
Office of Educational Research and Improvement 
EDUCATIONAL RESOURCES INFORMATION 
CENTER (ERIC) 

©--‘This document has been reproduced as 
received from the person or organization 
originating it. 

□ Minor changes have been made to 
improve reproduction quality. 



TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) 



Points of view or opinions stated in this 
document do not necessarily represent 
official OERI position or policy. 



The Power of Clarifying: A Comparative Analysis 
of Strategies that Strengthen Comprehension 



Shira Lubliner 

California State University, Hayward 



Annual Meeting of the American Educational Research Association 

April 12, 2002 




BIST COPY 



AVAILABLE 



Abstract 



This quasi-experimental study was designed to investigate the effects of cognitive strategy 
instruction on fifth grade students ’ reading comprehension and comprehension of words 
encountered during reading. Eight intact fifth grade classes comprised of approximately 34 
students each were randomly assigned to treatment groups (clarifying, questioning, combined 
questioning and clarifying, and control) for a five-week study. Strategy instruction took place 
during social studies based on the belief that that content area textbooks are the most difficult for 
children to read and comprehend (Chall, Jacobs, & Baldwin, 1990). Results indicated that the 
clarifying group ’s scores in reading comprehension and word comprehension were significantly 
higher than those in the control group, suggesting that clarifying strategy instruction positively 
affected fifth grade students ’ comprehension of texts and vocabulary encountered during 
reading. 
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A number of studies have demonstrated the effectiveness of cognitive strategies in 
improving children’s reading comprehension (National Institute of Health, 2000; Pressley et al., 
1995; Rosenshine, 1997). Research has not demonstrated, however, which specific strategy or 
combination of strategies contributes most powerfully to comprehension. This study was 
designed to measure the effectiveness of clarifying, a cognitive strategy that has not previously 
been investigated as a single strategy in the cognitive strategy literature. Clarifying was 
examined in conjunction with questioning, a strategy with a well-established research base 
(King, 1994; King & Rosenshine, 1993; Rosenshine, Meister & Chapman, 1996). The intent of 
the study was to examine whether clarifying contributes uniquely to students’ reading 
comprehension when instruction is properly designed and implemented. 

The conceptual framework for cognitive strategy research is based on the information 
processing theories of educational psychologists such as Gagne (1985), Bruner (1960, 1966), and 
Mayer (1981). Information processing theorists suggested that cognitive strategies are tools that 
learners use intentionally to regulate the learning process and that strategies can be taught to 
children to improve learning outcomes (Weinstein & Mayer, 1986). Reciprocal teaching 
researchers (Brown, 1985; Brown & Palincsar, 1985, 1986; Palincsar, 1983, 1984, 1985) 
expanded the conceptual framework underlying cognitive strategy research, grounding their 
studies on Vygotsky’s theory of social cognition (1978). They theorized that the interaction of a 
student with a more competent peer or teacher would result in the collaborative construction of 
meaning and improved reading comprehension. Palincsar and Brown developed reciprocal 
teaching, a method comprised of four cognitive strategies (questioning, clarifying, summarizing, 
and predicting) that was implemented successfully in studies with over 700 children. Rosenshine 
and Meister (1994) reviewed the reciprocal teaching literature, confirming that teaching children 
to implement cognitive strategies in text-based dialogues resulted in significant improvement in 
reading comprehension that was maintained over time. 

In a subsequent study, Rosenshine, Meister, and Chapman (1996) reviewed the 
questioning literature. They found that questioning, a single strategy, appeared to be equally 
effective in improving students’ reading comprehension as the combination of four reciprocal 
teaching strategies. The researchers raised the question as to what, if any, additional benefit was 
conferred by the other three strategies in the reciprocal teaching studies. Rosenshine et al. 

(1996) concluded that additional research investigating the effects of teaching different strategies 
and combinations of strategies is needed, providing a rationale for the current study. 

Little attention was paid to clarifying in the cognitive strategy research, however a 
parallel body of research explored the effects of vocabulary acquisition strategies on students’ 
reading comprehension. Research beginning in the early 1 980s suggested that strategic methods 
of vocabulary acquisition might be more effective than conventional vocabulary instruction. 
Camine, Kameenui, and Coyle (1984), for example, pointed out that two approaches are 
available to readers when they encounter unfamiliar vocabulary during reading. Unfamiliar 
words can be clarified utilizing a dictionary or students can utilize contextual clues to determine 
word meaning. According to the researchers, the word derivation approach has a number of 
advantages. Students are not dependent on a dictionary; the reading process proceeds with less 
disruption, and word derivation skills can be utilized during independent reading. 

A number of studies demonstrated that children could be taught to utilize vocabulary 
acquisition strategies, such as word derivation from context in relatively short interventions, but 
attempts to link vocabulary acquisition with improved reading comprehension have produced 
mixed results. Studies that documented increased reading comprehension achievement 
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following vocabulary acquisition instruction have typically been lengthy in duration and have 
stressed teaching the deep meaning of words. Beck and her colleagues (1982), for example, 
reported that a 19-week program of vocabulary instruction resulted in improved performance on 
transfer tests of vocabulary and reading comprehension. A follow-up study that spanned five 
months also resulted in improvements in vocabulary acquisition and reading comprehension 
(McKeown et al., 1983). 

Nagy (1988) synthesized the research literature on vocabulary and reading 
comprehension, pointing out that vocabulary instruction typically fails to provide in-depth word 
knowledge, resulting in little improvement in students’ reading comprehension. Repetition, 
according to Nagy, is a necessary component of effective vocabulary instruction, providing 
students with multiple exposures to new words in a variety of reading contexts. It is only after as 
many as ten encounters with a new word that students are able to transform a partially known 
word into the in-depth word knowledge necessary for reading comprehension. Nagy concluded 
that increasing reading volume, engaging students in meaningful processing of vocabulary 
during reading, and helping students make connections between new words and prior knowledge 
are the components of instruction that encourage the development of reading comprehension. 

Baker and his colleagues (1995) reviewed the literature on vocabulary acquisition, 
examining research findings on word learning strategies, the relationship between word 
knowledge and reading comprehension, and instructional approaches for developing students’ 
vocabulary. They noted a convergence of the research regarding the need for explicit strategy 
instruction, direct vocabulary instruction for a limited number of highly important words, and 
multiple exposures to unfamiliar vocabulary during the reading of connected texts. Baker et al. 
recommended instruction that involves students actively and helps to build associations with 
prior knowledge, stating that studies including these types of activities have been successful in 
improving students’ vocabulary. 

Tomesen and Aamoutse (1997) examined the effects of word derivation instruction on 
fourth grade students’ reading comprehension achievement. Results of the study indicated that 
students in the experimental group learned effective word derivation strategies and utilized these 
skills in reading texts. The students’ acquisition of word derivation strategies, however, did not 
lead to improvement in general reading comprehension. 

Fukkink and de Glopper (1998) conducted a meta-analysis examining the effectiveness of 
teaching students to derive word meaning from context. The researchers noted that most 
vocabulary growth by students could be attributed to incidental word learning from context and 
relatively little vocabulary growth was associated with direct instruction. Fukkink and de 
Glopper pointed out that incidental word learning from context is a highly inefficient method of 
vocabulary acquisition, suggesting the importance of developing strategies to promote students’ 
word derivation skills. Results of the meta-analysis were mixed, with ten out of a total of 19 
studies reporting significant effects on reading comprehension. 

Baumann et al. (2002) investigated the effects of instruction in vocabulary acquisition 
strategies including structural analysis and contextual analysis on students’ reading 
comprehension. Although students successfully learned vocabulary acquisition strategies, and 
transferred their skills to new vocabulary, no significant effect on reading comprehension 
achievement was documented. 

A number of studies evaluated the effects of cognitive strategy treatments of students of 
varying proficiency levels. McKeown found that lower ability children have more difficulty 
utilizing context clues in inferring word meaning than high ability children (1986). Stanovich 
(1986) also found that proficient readers are better at deriving word meaning from context, 
enabling them to acquire more vocabulary from reading than poor readers even when individual 
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differences in prior knowledge are controlled. Stahl (1990) examined the learner characteristics 
that foster or inhibit the derivation of word meaning from context, pointing out that students with 
lower levels of academic ability are less successful in performing word derivation activities. 
According to Stahl, low-achieving students’ limited background knowledge restricts their ability 
to benefit from contextual clues. Conversely, more proficient students have greater general 
knowledge that contributes to the construction of meaning during reading. Stahl concluded that 
high-achieving students are more capable of the inferences required to derive word meaning 
from context. Baker and his colleagues (1995) addressed the instructional implications of the 
studies they reviewed. They noted that efforts to teach low-achieving students word derivation 
strategies were often unsuccessful, suggesting that these students lacked the flexibility to utilize 
strategies effectively. Although results of previous studies (Anderson & Freebody, 1981; Baker 
et al.,1995; McKeown, 1985; Stahl, 1990; Stanovich, 1986) suggested that students with low 
reading ability were less proficient at deriving new words from context than high ability 
students, Nagy et al.’s (1985) study found no significant relationship between reading ability and 
word derivation. Nagy and his colleagues suggested that the use of a natural text and the wide 
range of target words in his study offered students of varying ability levels an equal opportunity 
to acquire new vocabulary. 

Rationale and research questions 

Questioning and clarifying are two distinct types of cognitive strategies. Questioning is 
clearly conceptualized in the cognitive strategy literature, and methods of questioning strategy 
instruction are explicitly defined. Moreover, teachers spend a predominant amount of 
instructional time asking questions (Durkin 1978), suggesting that familiarity with their own use 
of questioning may enable teachers to teach the questioning strategy effectively. It is possible 
that teacher comfort-level with questioning explains the successful implementation of this 
strategy that Rosenshine et al . ’ s ( 1 996) study detected. 

Conversely, clarifying has not been clearly defined in the cognitive strategy literature 
complicating teachers’ attempts to conceptualize and implement this strategy. The vocabulary 
acquisition studies appear to investigate the same underlying heuristic known as clarifying, 
however, teachers may not recognize vocabulary acquisition methods as the clarifying strategy. 
Rosenshine and Meister (1994) pointed out in their review of the reciprocal teaching studies that 
students rarely used the clarifying strategy. It is possible that inadequate teacher understanding 
of the clarifying strategy resulted in ineffective instruction and limited implementation by 
students. 

This study was designed to extend the boundaries of cognitive strategy research, 
integrating vocabulary acquisition studies into the cognitive strategy literature. The purpose of 
the study was to compare the relative effectiveness of three cognitive strategy treatments, 
questioning, clarifying, and a combination of questioning and clarifying, on students reading 
comprehension and comprehension of words encountered during reading. The study focused on 
the development of a powerful clarifying treatment that could be taught to students with a wide 
range of reading proficiency levels, positively affecting comprehension. The following research 
questions were addressed: 

■ Which treatment, questioning, clarifying, a combination of questioning and clarifying, or 
the control condition results in the highest mean scores in reading comprehension and 
word comprehension measured by the Gates MacGinitie and the researcher-designed 
tests? 

■ What is the relative effectiveness of questioning, clarifying, a combination of questioning 
and clarifying, or the control condition on students of varying proficiency levels? 



Method 

Participants 

The study was conducted in a large suburban school district in Northern California. The 
research design called for eight fifth grade classes, which were to be randomly assigned to four 
groups (two classes per group). Fifth grade teachers were recruited based on the following 
procedures: District support for the research project was secured and the elementary principals 
agreed to distribute recruitment letters to all of their fifth grade teachers. Eight teachers 
volunteered to participate in the study, however two withdrew at the last moment, leaving only 
six classes. The problem was resolved by including both social studies classes taught by two of 
the participating teachers. One teacher taught both sections of the combined strategy group and 
another teacher taught both control group classes. This arrangement resulted in a total of eight 
classes that were included in the study. 

The participating teachers (one male and five females) had earned California multiple 
subject teaching credentials, but no additional graduate degrees. At the time of the study they 
had been teaching for an average of six years, with a range of one to 10 years of teaching 
experience. The teachers work in four elementary schools in the district. The schools, located in 
a lower middle class community, have similar student populations that are approximately 75% 
White, 12% Latino, and 5% African American. A small percentage of the students are Asian or 
multi-ethnic. Approximately 1 3% of the students receive free or reduced price lunches, an 
indicator of low socio-economic status. Stanford 9 (SAT 9) composite reading scores from the 
previous spring indicate that student achievement averaged slightly below the 50 percentile in 
the participating schools. 

Design 



The study was quasi-experimental in design, comparing the effects of cognitive strategy 
treatments on students’ reading comprehension and word comprehension. Eight intact fifth 
grade classes comprised of approximately 34 students were randomly assigned to a treatment 
group (clarifying, questioning, questioning and clarifying combined) or to a control condition. 
Teachers were given training in the particular strategy or strategies they would be teaching and a 
complete packet of instructional materials to utilize with the students. In order to standardize the 
treatments as much as possible, reading materials and student activities were drawn from Early 
United States (Boehm et al., 2000), the social studies textbook recently adopted by the district 
and participating schools. Instruction in all of the classes was based on the same unit on the 
American Colonies, taught approximately three hours per week over a four-to-five-week period 
of time. (Weekly time allotments for social studies instruction varied among the schools, but 
students in all participating classes received a total of 12 hours of instruction during the 
experimental period.) 

Teachers in the three experimental groups were trained to use methods of explicit 
strategy instruction derived from the cognitive strategy literature (Gambrell et al., 1999; Pearson 
and Dole, 1985; Rosenshine, 1996, 1997). For example, teachers were asked to explain the 
purpose of strategy use, model new skills, scaffold student practice, and provide corrective 
feedback to students during cognitive strategy instruction. Packets for each treatment group 
included a teacher’s guide with lesson plans, instructions for strategy instruction and student 
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materials for 12 lessons. Two after-school training sessions averaging two hours in length were 
provided for teachers assigned to each of the treatment groups. 

Questioning Strategy Group. Questioning is defined as a cognitive strategy that entails 
generating text-based questions for the purpose of enhancing reading comprehension. Teachers 
assigned to the questioning strategy treatment were given training in teaching the questioning 
strategy and materials based on two sets of procedural prompts. Prompts included signal words, 
such as who, what, when, where, why, and how and prompts with elaborated question stems such 
as what would happen if... and why is this important? based on King’s (1994) model of guided 
questioning. Particular emphasis was placed on teaching students to generate main idea 
questions. Materials including cue cards and a series of instructional activities for teaching 
questioning skills were provided. Each lesson was based on questioning strategies that students 
implemented while reading the social studies textbook. The following is an example of a 
questioning strategy lesson taken from the Questioning Group Teacher’s Guide: 

Figure 1 

Questioning Strategy Lesson 

Session 6: Question stems: Connecting questions 
Purpose: 

The purpose of this lesson is to teach children to utilize question stems in asking connecting questions. Connecting 
questions activate prior knowledge, an important aid to learning. Using connecting question stems requires students 
to associate what they already know about a topic with new information. Connecting questions are the most 
complex and difficult for children to learn to utilize but deepen their understanding of a text tremendously (King, 
1994). 

Materials: 

Your materials include Cue Card #3 and worksheet Q-6. The content of the lesson is based on Chapter 5, Lesson 5 
in the Social Studies book. 
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Instruction: 

1 . Review the use of question stems to form questions. Explain to students that connecting question stems are 
the most difficult to use but they will help them the most. 

2. Pass out Cue Card #3 



QUESTIONING CUE CARD 3: CONNECTING QUESTIONS 

What is an example of ... ? 

How are ... and ... similar? 

How are ... and ... different? 

How does... affect...? 

What are the strengths and weaknesses of ...? 

What causes . . . ? 

What do you think would happen if ... ? 

How does ... tie in with ...that we learned before? 



3. Read aloud a fable such as The Tortoise and the Hare. 

4. Model the use of connecting question stems with the fable. 

5. Select student volunteers to take turns play the role of teacher. Each volunteer reads a paragraph and asks 
connecting questions which the class answers. 

6. Model the use of connecting questions with the social studies book. 

7. Ask students to work in pairs, generating and answering questions based on Chapter 5, lesson 5 of the 
social studies book 

8. Go over the directions to worksheet Q-6. Have students complete the worksheet for homework. 

9. Following day: check the worksheets for understanding 



Questioning strategy instruction was based on a series of cue cards containing question 
words and question stems. Initially, students practiced the generating questions with simple texts 
such as nursery rhymes and fables and then implemented the strategy with their social studies 
books. 



Clarifying Strategy Group. Clarifying is defined as an umbrella term for cognitive 
strategies used to identify the meaning of unfamiliar words or phrases in a text. The clarifying 
strategy teachers received training in two clarifying strategies identified in the vocabulary 
acquisition research, the derivation of word meaning from context and structural analysis 
(Baumann et al., 2002; Fukkink & de Glopper, 1998; Nagy, 1988; Nagy et al., 1985; Stahl, 1983, 
1990, 1999). An additional strategy, the intentional use of memory for word learning was added 
to strengthen the effects of the clarifying treatment. Teachers were taught to provide explicit 
instruction in each clarifying strategy, to guide student practice, and to teach students to 
implement the strategies with the social studies textbook. Teachers were trained to use 
researcher-developed materials such as cue cards designed to scaffold clarifying strategy use and 
to encourage flexible strategy implementation during reading. An example of a clarifying cue 
card is provided in Figure 2. 

Figure 2 

Clarifying Cue Card 
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WHEN YOU FIND A WORD YOU DON f T UNDERSTAND TRY THE FOLLOWING STRA TEGIES: 

CONSIDER THE CONTEXT: Look at the information in the sentence and the whole paragraph and see 
if you can figure out the meaning of the word. 

SUBSTITUE A SYNONYM: When you think you know what the word means try putting a word with a 
similar meaning in the sentence. Does it make sense? 

STUDY THE STRUCTURE: Do you know the root word? Does the word have a prefix or suffix that 
you know? Try to use clues in the word to figure out the meaning. 

MINE YOUR MEMORY: Have you ever seen this word before? Can you remember what it meant? 

ASK AN EXPERT : Does someone in your group know what the word means? Can you figure it out 
together? 

PLACE A POST-IT: If you can’t figure out the meaning of the word put a post-it in the book and check 
with the teacher or look it up in the dictionary later. 



The following is an overview of clarifying strategy instruction based on the cue card 
(Figure 2): Students were taught to use context, structure, and memory as strategies for 
clarifying unfamiliar words encountered during reading. They were given the cue card and 
reminded that the strategies listed would help them read more proficiently. Consider the Context 
was the first clarifying strategy listed on the cue card. Context was defined broadly and students 
were given explicit instructions on how to be a “context detective”. Substitute a Synonym was 
taught as a self-checking prompt to help students determine whether their clarification of an 
unfamiliar word made sense. Students were taught to Study the Structure of unfamiliar words, 
attempting to identify inflected endings such as “ing” and “ed”, roots, suffixes, and prefixes that 
provide clues to word meaning. Instruction in Greek and Latin roots and activities such as word 
sorts were provided to help students become familiar with families of words found in the social 
studies textbook. Mine Your Memory was a strategy used to prompt recall of unfamiliar words 
encountered during reading. Rather than skipping a difficult word students were asked to try to 
remember if they had seen or heard the word before. In many cases they were able to recall 
previous encounters with the word, leading to a partial construction of word meaning. 

Ask an Expert was a prompt, designed to encourage the collaborative construction of 
meaning by students, working in pairs or small groups. Place a Post-it was also a prompt to be 
used if none of the clarifying strategies were successful. Students were taught to mark the 
unknown word with an adhesive note so that it could be looked up in the dictionary or discussed 
with the teacher when reading was completed. 

Social studies textbooks are difficult for children to read and understand due, in part, to 
the large number of conceptually difficult words included in the texts. In order to address this 
problem, clarifying and combined strategy teachers were provided with activities to encourage 
the acquisition of conceptually challenging vocabulary found in American Colonies Unit of the 
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social studies textbook. The vocabulary and reading passages on the researcher-designed tests 
were NOT included in the instructional materials. 

Figure 3 includes an example of Stoplight Vocabulary, an activity designed to teach the 
conceptually challenging words found in an instructional unit. Challenging words, selected by 
the teacher, are listed next to traffic signals that the children color according to their current level 
of word knowledge. A red-light word is one that is completely unknown to the student and is 
colored red. Children color the traffic signal yellow to indicate partial word knowledge and 
green if the word is well known and can be used independently in speaking and writing. 

Figure 3 

Sample Stoplight Vocabulary Words from the Social Studies Unit on the American Colonies 



Establish 




Expedition 



Red Yellow Green 




Students in the clarifying and combined strategy groups were taught to use Stoplight 
Vocabulary as a metacognitive device, monitoring their own word knowledge over the course of 
the social studies unit. Students whose initial traffic signals included many red-light words were 
encouraged to color the appropriate yellow and green signals as word learning occurred. This 
method was designed to encourage self-monitoring of word learning by the students. 
Additionally, explicit vocabulary instructional activities, such as Stoplight Vocabulary , were 
intended to reinforce the effectiveness of the memory strategy, providing multiple exposures to 
challenging and important social studies words. 

Questioning and Clarifying Group. Teachers assigned to the questioning and clarifying 
combined strategy treatment were given training and a condensed version of the instructional 
materials for teaching both strategies. Strategy instruction for the combination group was rotated 
between the two strategies on a weekly basis. The rationale for rotating the two strategies was to 
maintain students’ skills in both strategies, lessening the likelihood that the strategy taught first 
would be forgotten by the time the posttest was administered. 

Control. The teacher who was assigned to the control condition stated that she did not 
use cognitive strategies with the students in her two social studies classes. She agreed to base 
her instruction on the social studies textbook teacher’s guide and to refrain from introducing 
cognitive strategies during the duration of the study. During a post-study interview, however, 
she acknowledged initiating a program of cognitive strategy instruction, based on her 
interpretation of the book, Mosaic of Thought (Keene & Zimmermann, 1997). The control 
group teacher explained that she taught the students to use summarizing, predicting, visualizing, 
and rereading strategies during social studies instruction and she required parental monitoring of 
strategy practice at home. According to the documentation that she provided, the two control 
group classes appear to have received more intensive strategy instruction than any of the 
experimental groups that were limited by the research design to 12-hour strategy treatments. The 
control group, subsequently renamed the control (rogue) group, did not provide the carefully 
structured comparison of experimental group achievement that had been anticipated. 

Instrumentation 
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Three types of reading comprehension and word comprehension tests were utilized in the 
data analysis. The covariates included the SAT 9 reading comprehension, vocabulary, and 
language subtests, form K of the Gates MacGinitie Reading Test (MacGinitie, et al., 2000), and 
the researcher-designed pretest of reading comprehension and vocabulary. The dependent 
variables included form L of Gates MacGinitie Reading Test and the researcher-designed test of 
reading comprehension and vocabulary. The SAT 9 scores (administered in Spring, 2000) were 
provided by the school district. The Gates MacGinitie and the researcher-designed reading 
comprehension test were administered on two separate occasions prior to the beginning of 
instruction. One testing session was provided for the Gates MacGinitie and one day for the 
researcher-designed test. Two sessions were also devoted to test administration the week after 
the instructional period ended. Forty-five-to-fifty minutes were required for each testing session. 

The decision to use the Gates MacGinitie Reading test and a researcher-designed 
assessment as dependent variables was based on the instrumentation used in highly regarded 
cognitive strategy studies (Rosenshine & Meister, 1994; Rosenshine et al., 1996). Because 
standardized tests such as the Gates-MacGinitie are often insensitive to instructional treatments, 
dependent variables more closely aligned to instruction were also needed for the study (Nitko, 
1995; Shepard, 1994; Shepard et al., 1996). Two forms of a researcher-designed test of reading 
comprehension and word comprehension, were developed and administered to the participating 
students. 

The researcher-designed tests were based on reading passages from America Will Be 
(Armento et al., 1991), a fifth grade social studies textbook that is no longer in use in the District. 
Topics were selected that are not typically taught until late in the school year to maximize the 
chances that test scores would reflect reading comprehension, rather than prior content 
knowledge. Eight reading passages (two per topic) were selected from America Will Be . Topics 
included mechanization, immigration, industrialization, and emancipation. A readability 
analysis was conducting, indicating that the passages were at a ninth-to-tenth grade reading level 
(Fry, 1977, McLaughlin, 1969). The passages were revised, lowering the readability to an eighth 
grade level (as low as possible while preserving content). Passage length was standardized at 
approximately 200 words and multiple-choice items, measuring reading comprehension and 
word comprehension were developed for each passage. Item sets, comprised of a reading 
passage and 10 multiple choice items were matched for similar content and organized into two 
parallel forms of the test. Each form of the test included a total of four item sets resulting in a 
total of 40 test items. Thirty items on each form of the test were designed to measure reading 
comprehension and 10 items were designed to measure students’ proficiency at determining the 
meaning of unfamiliar words encountered during reading. Although the items measuring reading 
comprehension and word comprehension were based on the same reading passages they were 
designed to measure different learning outcomes. Therefore, the reading comprehension items 
and word comprehension items were divided into two instruments (reading comprehension test 
and word comprehension test) for the purposes of data analysis. 

Test development included a number of procedures intended to provide evidence of 
content-related validity. The articles and corresponding item sets were reviewed independently 
by panel of literacy experts that included reading specialists, fifth grade teachers, and a professor 
of academic literacy. The members of the panel agreed that the articles were written at a 
readability level consistent with textbooks fifth graders are expected to read and comprehend. 
The experts reviewed each test item, concluding that the items measured important reading and 
word comprehension skills. They identified several items which included ambiguous distractors 
and made suggestions for revision. The researcher-designed instruments were revised to 
conform with the suggestions by the literacy experts. The revised instruments were then pilot 
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tested with two fifth grade classes in the same school district. An overall correlation of .82 
between the two forms of the pilot test demonstrated the consistency of test results and provided 
evidence of the reliability of the researcher-designed instruments. 

Once the study was completed the resulting scores were analyzed and the correlation 
between the researcher-designed tests and the Gates MacGinitie were evaluated for evidence of 
validity. The correlation between the comprehension tests was .72, an indication that the 
researcher-designed test measured similar comprehension skills as the standardized Gates 
MacGinitie comprehension test. The correlation between the researcher-designed word 
comprehension test and the vocabulary component of the Gates MacGinitie was .66. This 
correlation suggests a substantial amount of shared variance between the word comprehension 
assessment and the standardized vocabulary test, despite the fact that the tests measure somewhat 
different word comprehension proficiencies. 

The following is a brief excerpt of one of the 200-word articles and two items from the 
researcher-designed test. The first item was designed to measure word comprehension and the 
second item measures reading comprehension skills: 

Figure 4 

Excerpt of the Researcher-Designed Test 



Article One (The second of four paragraphs on mechanization) 

Farmers who could not afford to buy new machines looked for something else to do. They saw the city as a place to 
start over. Cities promised good jobs and a chance to succeed. Busy city life drew many Americans away from their 
farms and small towns. As a result, cities grew at an amazing rate. In 1860 only about 6 million people lived in U.S. 
cities. By 1920 more than 54 million people were city dwellers. 

(Two of ten questions based on Article One) 

1 . The word dwellers means: 

a) people who live in a place 

b) mechanical tools 

c) people without jobs 

d) factory workers 

2. What evidence (proof) is given that cities grew very quickly? 

a) U.S. cities grew at an amazing rate 

b) in 1920 6 million people lived in U.S. cities 

c) U.S. cities grew from 6 million in 1860 to 54 million in 1920 

d) 54 million people lived in U.S. cities in the year 1860 



Note the high readability and complex ideas that characterize this social studies passage, 
adapted from America Will Be (Armento et ah, 1991). The comprehension items on the 
researcher-designed test were designed to be consistent with other challenging assessments that 
require children to comprehend complex ideas and draw inferences from a text (Cunningham & 
Allington, 1999). 

Conversely, the vocabulary items were designed to be different from items measuring 
simple word knowledge included on most standardized tests. For example, Item 1 on the test 
measures a child’s ability to make sense of the word “dwellers” based on clues in the text, in 
addition to assessing whether the word “dwellers” is included in the child’s lexicon. 




13 



12 



Results 



The Gates MacGinitie tests of reading and vocabulary (form L) and the researcher- 
designed reading comprehension and word comprehension posttests were administered to 
students at the end of the five-week study. A flu epidemic during the testing period resulted in 
frequent absences, reducing the number of student scores that could be obtained. Complete data 
based on SAT 9 scores from the prior year and all four pretests and posttests administered during 
the study were collected from a total of 193 students. 

A multiple analysis of covariance (MANCOVA) was used to analyze the data. This 
method was chosen to account for the shared variance between the highly correlated dependent 
variables. (All MANCOVA assumptions were reasonably met.) Analysis of the Gates 
MacGinitie posttest results did not reveal significant differences between mean strategy group 
scores and those of the control (rogue) group. These results appeared to be consistent with prior 
research documenting the insensitivity of standardized tests to brief instructional interventions 
(Nitko, 1995; Shepard, 1994; Shepard et al., 1996). The results discussed in this paper will 
focus on researcher-designed tests. 

Results of the multivariate test indicated significant between group differences (p<.01) 
that were small-to-moderate in magnitude (eta 2 = .04). The observed power level of .99 was 
very high, suggesting a sensitive research design and a strong likelihood that all significant 
between-group differences were detected by the analysis. The MANCOVA also produced 
univariate results and pairwise comparisons for the dependent variables that were assessed using 
Tukey’s Least Significant Differences. MANCOVA results (pairwise comparisons) comparing 
treatment group and control (rogue) group performance indicated significant differences favoring 
the clarifying group on both dependent variables. An effect size (Cohen’s d) was calculated to 
evaluate the practical significance of each comparison. 

Results of the reading comprehension test (30 multiple-choice items) indicated an overall 
mean of 17.99 with a standard deviation of 5.66 (Table 1). The lowest score was three and the 
highest score was 28. The clarifying group earned the highest mean scores following covariate 
adjustment. The adjusted mean scores of the control (rogue) group were the lowest of the four 
groups. 

Table 1 

Descriptive Data: Reading Comprehension Test 



Group 


N 


Adjusted 

Mean 


Standard 

Deviation 


Clarifying 


44 


18.55 


5.59 


Questioning 


44 


18.13 


5.51 


Combined 


53 


17.73 


5.98 


Control (rogue) 


52 


16.99 


5.39 



Results of the pairwise comparisons (Table 2) indicate that the mean score of the 
clarifying group was significantly higher than that of the rogue (control) group. The test of 
practical significance (Cohen’s d ) indicated a small effect in favor of the clarifying group (d = 
.28). None of the other pairwise comparisons were significant. 
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Table 2 



Significant Pairwise Comparisons for the Reading Comprehension Test 



Higher scoring 


Adjusted 


Lower scoring 


Adjusted 


d 


Group 


Mean 


Group 


Mean 




Clarifying 


18.55 


Control (rogue) 


16.99 


0.28* 



* p < .05 



MANCOVA results were used to compare word comprehension scores on the researcher- 
designed test (10 multiple-choice items). The mean score for the group (193 students) was 5.55 
with a standard deviation of 2. 1 8. The lowest score was zero and the highest score was 1 0. 

The clarifying group’s adjusted mean score was higher than the mean scores of the other 
groups (Table 3). Further, the clarifying group’s standard deviation was the smallest suggesting 
that student performance in the clarifying group had less variance than that of students in other 
groups. 



Table 3 

Descriptive Data for the Word Comprehension Test 



Group 


N 


Adjusted Mean 


Standard Deviation 


Clarifying 


44 


6.32 


1.71 


Control (rogue) 


52 


5.51 


1.94 


Combined 


53 


5.30 


2.27 


Questioning 


44 


5.10 


2.34 



The clarifying group received significantly higher scores on the word comprehension test 
than the questioning, combined, and control (rogue) groups (Table 4). Tests of practical 
significance (Cohen’s d) were calculated indicating moderate effects, ranging from .42 to .62, in 
favor of the clarifying group. 



Table 4 

Significant Pairwise Comparisons for the Word Comprehension Test 



Higher Scoring Group 


Lower Scoring Group 


d 


Clarifying 


Questioning 


0.62** 


Clarifying 


Combined 


0.53** 


Clarifying 


Rogue (Control) 


0.42** 



** p<.01 




15 



14 



In order to determine the relative effects of cognitive strategy treatments on students of 
differing proficiency levels, the 193 participating students were divided into four quartile groups 
based on SAT 9 Total Reading scores (Spring, 2000). 

Figure 4 

Interaction Graph for the Reading Comprehension Test 



r 






Reading Comprehension 




-Quartile 1 —©—Quartile 2 



-Quartile 3 



-Quartile 4 



The interaction graph for the reading comprehension test (Figure 4) shows differing 
patterns of achievement for the four quartile groups. Unlike the other groups, the highest 
quartile students appeared to receive the greatest benefit from questioning strategy instruction. 
Students in the middle quartiles (two and three) appeared to benefit most from clarifying strategy 
instruction, demonstrating substantially higher scores than students in the other groups. The 
lowest quartile students showed minimal achievement differences across treatment groups. 
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Figure 5 

Interaction Graph for the Word Comprehension Test 




The interaction graph for the word comprehension test (Figure 3) indicates higher levels 
of achievement by students in the clarifying group in all four quartiles, suggesting that students 
of all achievement levels benefited from clarifying strategy instruction. A parallel pattern of 
achievement favoring clarifying over questioning in the first, second, and third quartile groups is 
evident. The students in the highest quartile, however, had a less pronounced pattern of 
achievement favoring the clarifying group. 

When results of the word comprehension reading comprehension tests are compared a 
similar pattern of questioning group achievement can be noted in the highest quartile. The 
results of the quartile group analyses suggest that highly capable students may have benefited 
more from questioning strategy instruction than less capable students. 



Discussion and Conclusions 

This study began with an observation that clarifying was poorly defined in the cognitive 
strategy literature and appeared to be ineffectively utilized in reciprocal teaching interventions. 
An investigation of the vocabulary acquisition literature yielded many helpful ideas resulting in 
the development of the clarifying treatment that was examined in this study. The discussion 
section of this paper focuses on the overall effectiveness of clarifying strategy instruction, the 
effectiveness of clarifying strategy instruction for children of differing proficiency levels, and the 
practicality of implementing clarifying instruction in schools. Topics of discussion also include 
an analysis of the results of the combined strategy treatment, limitations and directions for future 
research. 

The Effectiveness of Clarifying 

Students in the clarifying group earned higher adjusted mean scores in both dependent 
measures than students in the questioning, combined, and control (rogue) groups. These results 
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suggest that clarifying instruction was effective in increasing students reading comprehension 
and word comprehension skills relative to other treatments. It is important to note that the 
passages on the researcher-designed test were unfamiliar to the students demonstrating that they 
were able to successfully transfer clarifying skills taught in class to comprehension of the test 
passages. 

The vocabulary acquisition literature does not provide convincing evidence that brief 
instructional interventions positively effect reading comprehension achievement, raising the 
question as to why this particular study, only 12 hours in duration, produced significant results. 
The instructional intervention in this study was based on three distinct clarifying strategies 
(memory, context and structure) and an emphasis on developing cognitive flexibility related to 
word learning. Although it is not possible to identify which of these factors or combination of 
factors was most salient in the results, it appears that the clarifying treatment was more varied, 
possibly producing a stronger effect than the instruction included in most studies. 

The inclusion of memory as a cognitive strategy may have contributed to the 
effectiveness of the clarifying treatment. Teaching children intentional use of memory in word 
learning was based on research highlighting the importance of repeated exposures to new words 
in the acquisition of vocabulary (Baker, 1995; Nagy, 1988; Nagy et al., 1985, 1987). Rather than 
leaving word learning during reading to chance, the clarifying treatment was designed to teach 
word retrieval strategies systematically as a means of developing students’ word learning 
proficiency. The effectiveness of this strategy was underscored in a post-study discussion with 
one of the clarifying classes. When students were asked which strategy was most helpful to 
them, the majority responded that they relied on “mine your memory” most frequently in 
attempting to clarify unknown words. Individual children gave examples of hard words that they 
remembered from other books, television shows, and conversations with adults. Children 
explained that prompting from the clarifying cue card helped them realize that they “kind of 
knew” many challenging words encountered dining reading. 

In addition to the inclusion of three distinct strategies, the clarifying treatment was 
designed to broaden the way each strategy was used. Instruction in word structure, for example, 
included inflected endings, prefixes, suffixes, and Greek and Latin roots. Unlike studies that 
focused on a single morphemic feature such as prefixes (Baumann et al., 2002) clarifying 
instruction in this study exposed children to a variety of structural clues. The intent was to 
provide structural clues that could be used by students with a wide range of reading proficiency 
levels. The usefulness of this approach was confirmed in post-study interviews with clarifying 
and combined strategy teachers. Teachers mentioned that prior to clarifying instruction, 
struggling readers often skipped lengthy words without realizing that they already knew the root 
words. Learning to identify familiar words with inflected endings such as “ed” and “ing” 
appeared to be helpful to these impaired readers, expanding their willingness to engage with 
texts. More proficient students, on the other hand, utilized a range of structural clues that had 
been taught, relying on prefixes, suffixes, and Greek and Latin roots to infer word meaning. 

Another factor that may have contributed to the positive results of the clarifying 
treatment was the fact that instruction was designed to develop students’ cognitive flexibility in 
reference to strategy implementation. Children were taught how each strategy could be used and 
were given opportunities to practice successful strategy implementation. Then students were 
shown examples of how each strategy failed to work and were prompted to try additional 
strategies to infer word meaning. Students were also encouraged to verify their understanding of 
word meaning by crosschecking strategies and thinking about whether a particular clarification 
“makes sense”. This method was designed to increase cognitive flexibility and persistence, 
while providing multiple exposures to target words. 
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Research suggests that incidental word learning during reading is the major source of 
vocabulary growth (Fukkink & de Glopper, 1998; Nagy, 1984, 1985; Nagy et al„ 1987; 

Stanovichl 1986; Swanbom & de Glopper, 1999). Incidental word earning however, is a highly 
inefficient process, providing only a .05 chance that a word will be earned from a single 
encounter (Nagy, 1985; Nagy et al„ 1987). Transforming incidental word learning during 
reading into deliberate word learning through use of a clarifying cue card was intended to 
increase the efficiency of the process. Even when a student was unsuccessful in clarifying an 
unknown word encountered during reading, use of the cue card provided up to six exposures to 
the word as attempts were made to implement each clarifying strategy or prompt This process is 
distinctly different from skipping an unfamiliar word, a more common practice that provides 
only a single exposure to the word and limited opportunities for word learning. 

It is also possible that the positive effect of the clarifying treatment can be linked to the 
context of instruction and assessment. Students learned to implement clarifying strategies while 
reading social studies textbooks. The reading comprehension and word comprehension tests 
were based on social studies passages that were similar to the texts used for instruction. The 
students may have learned strategies that were specifically applicable to social studies, readily 
transferring strategic skills to unfamiliar texts of the same type. It would be interesting to 
investigate the extent to which clarifying strategies taught in a social studies context transfer to 
other content area texts such as science books and across genres to include fictional texts such as 

novels and poetry. 

The Effectiveness of Clarifying For Students of Differing reading proficiency levels 

The analysis of research results by quartile group also has important educational 
implications. The fourth quartile group (the highest achieving group) appeared to benefit more 
from questioning strategy instruction than students in the three lower quartiles. Generating a 
question in response to reading imposes intense cognitive demands on students, requiring that a 
series of information processing tasks be carried out rapidly and efficiently (Kintech & van Dijk, 
1978). The text must be understood as it is read and the gist of the text must be held in working 
memory while the student identifies important ideas, and articulates a relevant question. It is 
possible that highly capable students benefit from questioning strategy instruction because they 
are better able to cope with the cognitive demands that self-generated questioning entails. 

Differing achievement results may reflect the difficulty in mastering the questioning strategy and 
in transferring questioning strategy skills to reading comprehension and word comprehension 
tasks It is also possible that mastery of the questioning strategy requires a lengthier period of 
instruction if it is to be of value to lower performing students. Results of the study raise a 
concern that questioning strategy instruction may widen the achievement gap between high and 
low performing students unless it is carefully designed to accommodate diverse learning needs. 

The results of the quartile analysis suggest that clarifying strategy instruction benefited a 
wider range of students than questioning. Clarifying group students in the first, second, and third 
quartiles earned higher reading comprehension scores than students in the other groups. 

Clarifying was most effective treatment for students in all four quartile groups on the word 
comprehension test. The effectiveness of clarifying for low performing students was so 
pronounced in the word comprehension test that the lowest quartile clarifying students outscored 

questioning group students in the second and third quartiles. 

The effectiveness of clarifying may have resulted from the range of strategies that were 
available to students of different proficiency levels. An analysis of these strategies illustrates the 
range of cognitive demands that students encounter when attempting to infer word meaning from 
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context, structure, or memory. “Mine your memory” appears to be the most accessible 
clarifying strategy, implemented effectively by children of varying levels of reading proficiency. 
Although language exposure varies a great deal, most fifth grade students encounter vast 
numbers of words in oral and written contexts. Accessing prior word knowledge and learning to 
apply partial word understanding may be a relatively easy strategic task. 

Using structure to clarify unfamiliar words also draws heavily on prior knowledge. 
Children must recall the meaning of roots, prefixes, and suffixes that are found in unfamiliar 
words. Clarifying based on structural analysis requires students to flexibly manipulate word 
parts and to draw inferences regarding the meaning of words encountered during reading. 
Structural analysis appears to be a somewhat more challenging clarifying strategy for children to 
implement than memory. 

The derivation of word meaning from context appears to be the most complex clarifying 
strategy, requiring similar skills to the questioning strategy. Students must understand the gist of 
text and draw inferences about an unfamiliar word. Context must be applied broadly and 
flexibly, often requiring students to examine word meaning in relationship to the sentence, 
paragraph, and text as a whole. 

Prior research suggests that children with lower levels of academic proficiency have 
difficulty utilizing word comprehension strategies and are often unsuccessful in learning to 
derive word meaning from context (Anderson & Freebody, 1981; Baker et al.,1995; McKeown, 
1985; Stahl, 1990; Stanovich, 1986). This study, however, supports research by Nagy (1985) 
and Tomesen and Aamoutse (1997) demonstrating that students of varying proficiency levels, 
benefit from strategic vocabulary instruction. Nagy (1985) suggested that the use of a natural 
text and a range of target words offered students of varying ability levels an equal opportunity to 
acquire new vocabulary, factors that may have been salient in this study as well. 

The Effectiveness of Combined Strategy Instruction 

The results of the study demonstrate the effectiveness of clarifying strategy instruction, 
but an important question remains: Why didn’t a treatment based on two powerful cognitive 
strategies, questioning and clarifying, produce the highest level of student achievement? The 
combined strategy group students earned lower mean scores than either single strategy group 
(questioning or clarifying) in reading comprehension. The combined group’s scores were lower 
than all of the other groups on the word comprehension test. 

It is likely that the brief duration of the study was a factor in these results, a finding with 
important implications for classroom instruction. As other researchers have noted, effective 
cognitive strategy instruction requires extensive periods of time (Pressley et al., 1995). The 
combined group students may have been overwhelmed by the teacher’s attempt to teach them 
two complex strategies in twelve hours, resulting in inadequate learning of either strategy. It is 
also possible that rotating strategies each week confused the students and decreased the 
effectiveness of instruction. 

Practicality of Clarifying Strategy Instruction in Schools 

This study suggests that clarifying strategy instruction can be readily implemented in 
schools. Clarifying group teachers received only four hours of training prior to implementing 
the clarifying curriculum with their students. Children were given a total of 12 hours of 
clarifying strategy instruction during regular social studies lessons. The positive effect of this 
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brief intervention on reading comprehension and vocabulary achievement suggests that 
clarifying strategy instruction is a practical and effective means for improving student literacy. 

It is interesting to note that teacher quality appeared to matter less than the instructional 
method in the outcome of the study. All teachers were observed several times during the study, 
and the quality of instruction was informally evaluated. The combined strategy teacher appeared 
to be highly effective compared to the other participating teachers but his students’ scores were 
lower than expected. One of the clarifying group teachers was a first-year novice who appeared 
to be struggling to master basic instruction and management skills. Contrary to expectations, the 
novice teacher’ s students received high scores on both tests. The strong performance of the 
novice clarifying teacher’s students suggests that clarifying can be readily taught to teachers with 
a range of skills and experience, a finding with important implications for educational practice. 

Limitations and Directions for Future Research 

Several limitations must be considered in interpreting research results. Data analysis was 
based on researcher-designed instruments. The tests were carefully designed to address 
reliability and validity standards, however, additional work may be needed to improve the 
quality of these instruments. The word comprehension test, for example, included 1 0 items, a 
number that may not have been sufficient for reliable results. An insufficient number of word 
comprehension items may have resulted in a ceiling effect, limiting the sensitivity of this 
instrument to the achievement of higher performing students. Validation of the word 
comprehension test was particularly difficult because it was constructed to measure skills not 
normally included on vocabulary tests. Additional research is needed to design and validate 
instruments that measure students’ ability to apply clarifying strategies to reading comprehension 
and word comprehension tasks. 

Additional limitations regarding the findings of this study have been previously noted. 

The lack of a true control group due to the unmonitored strategy instruction provided by the 
control (rogue) teacher makes comparisons of group achievement difficult to interpret with 
confidence. The fact that one teacher taught both control (rogue) classes and one teacher taught 
both combined strategy classes may have accentuated the confounding variable of instructional 

effectiveness and distorted research results. 

Time constraints, particularly associated with the combined strategy group, may have 
also affected research results. The likelihood that time constraints interfered with the progress of 
the combined strategy students raises the following questions that should be addressed by future 
research: What is the threshold for learning a single cognitive strategy? Is a combined strategy 
treatment (questioning and clarifying) more effective than single strategy treatments when 
instructional time is adequate? 

Results of this study raised intriguing questions regarding effectiveness of questioning 
and clarifying strategy instruction for students of varying reading proficiency levels. It is 
important to investigate this issue further, to determine which factors are salient in students 
ability to benefit from cognitive strategy instruction. Do high performing students really benefit 
more from questioning strategy instruction than other students or are they simply able to learn 
the strategy more quickly than their less proficient peers? Additional research is needed to 
answer the questions that have been raised and to carefully examine the effectiveness of the 
clarifying treatment. 

This study was designed to extend the cognitive strategy literature; evaluating which 
strategy or combination of strategies is most effective in improving children s reading 
comprehension and comprehension of words encountered during reading. The intent was to 
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demonstrate that clarifying strategies are powerful tools that can be readily taught in norma 
instructional contexts. The results of the study suggest that clarifying strategy instruction has the 
potential to strengthen student literacy, providing comprehension tools that can be used 
effectively by students of all proficiency levels. 
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